Knowledge Acquisition by Demonstration

Traditional knowledge acquisition tools rely either on conversational textual interfaces or on graphical editing of rule and description structures. Both these methods divorce the user from the actual concrete problem-solving situations where the knowledge is created and applied. We show how a programming by demonstration system can be used as a knowledge acquisition tool by allowing a user to demonstrate procedures interactively on images taken from a video of the procedure being performed in the real world. Graphical annotation and speech input and output are used to provide declarative information that informs the system about objects in the scene, and how to interpret the user’s actions.